Inertia-Constrained Pixel-by-Pixel Nonnegative Matrix Factorisation: a Hyperspectral Unmixing Method Dealing with Intra-class Variability

Blind source separation is a common processing tool to analyse the constitution of pixels of hyperspectral images. Such methods usually suppose that pure pixel spectra (endmembers) are the same in all the image for each class of materials. In the framework of remote sensing, such an assumption is no more valid in the presence of intra-class variabilities due to illumination conditions, weathering, slight variations of the pure materials, etc... In this paper, we first describe the results of investigations highlighting intra-class variability measured in real images. Considering these results, a new formulation of the linear mixing model is presented leading to two new methods. Unconstrained Pixel-by-pixel NMF (UP-NMF) is a new blind source separation method based on the assumption of a linear mixing model, which can deal with intra-class variability. To overcome UP-NMF limitations an extended method is proposed, named Inertia-constrained Pixel-by-pixel NMF (IP-NMF). For each sensed spectrum, these extended versions of NMF extract a corresponding set of source spectra. A constraint is set to limit the spreading of each source's estimates in IP-NMF. The methods are tested on a semi-synthetic data set built with spectra extracted from a real hyperspectral image and then numerically mixed. We thus demonstrate the interest of our methods for realistic source variabilities. Finally, IP-NMF is tested on a real data set and it is shown to yield better performance than state of the art methods

Spectral Optimization of Airborne Multispectral Camera for Land Cover Classification: Automatic Feature Selection and Spectral Band Clustering

Hyperspectral imagery consists of hundreds of contiguous spectral bands. However, most of them are redundant. Thus a subset of well-chosen bands is generally sufficient for a specific problem, enabling to design adapted superspectral sensors dedicated to specific land cover classification. Related both to feature selection and extraction, spectral optimization identifies the most relevant band subset for specific applications, involving a band subset relevance score as well as a method to optimize it. This study first focuses on the choice of such relevance score. Several criteria are compared through both quantitative and qualitative analyses. To have a fair comparison, all tested criteria are compared to classic hyperspectral data sets using the same optimization heuristics: an incremental one to assess the impact of the number of selected bands and a stochastic one to obtain several possible good band subsets and to derive band importance measures out of intermediate good band subsets. Last, a specific approach is proposed to cope with the optimization of bandwidth. It consists in building a hierarchy of groups of adjacent bands, according to a score to decide which adjacent bands must be merged, before band selection is performed at the different levels of this hierarchy

Analysis of the performance of the TES algorithm over urban areas

International audienceThe temperature and emissivity separation (TES) algorithm is used to retrieve the land surface emissivity (LSE) and land surface temperature (LST) values from multispectral thermal infrared sensors. In this paper, we analyze the performance of this methodology over urban areas, which are characterized by a large number of different surface materials, a variability in the lowest layer of the atmospheric profiles, and a 3-D structure. These specificities induce errors in the LSE and LST retrieval, which should be quantified. With this aim, the efficiency of the TES algorithm over urban materials, the atmospheric correction, and the impact of the 3-D architecture of urban scenes are analyzed. The method is based on the use of a 3-D radiative transfer tool, TITAN, for modeling all of the radiative components of the signal registered by a sensor. From the sensor radiance, an atmosphere compensation process is applied, followed by a TES methodology that considers the observed scene to be a flat surface. Finally, the retrieved LSE and LST are compared with the original parameters. Results show the following: First, the TES algorithm used reproduces the LSE (LST) of urban materials within a root-mean-square error (rmse) of 0.017 (0.9 K). Second, 20% of uncertainty in the water vapor content of the total atmosphere introduces an rmse of 0.005 (0.4 K) for the LSE (LST) product. Third, in a standard case, the 3-D structure of an urban canyon leads to an rmse of 0.005 (0.2 K) for the LSE (LST) retrieval of the asphalt at the bottom of the scene

p3^3VAE: a physics-integrated generative model. Application to the semantic segmentation of optical remote sensing images

The combination of machine learning models with physical models is a recent research path to learn robust data representations. In this paper, we introduce p$^3$VAE, a generative model that integrates a perfect physical model which partially explains the true underlying factors of variation in the data. To fully leverage our hybrid design, we propose a semi-supervised optimization procedure and an inference scheme that comes along meaningful uncertainty estimates. We apply p$^3$VAE to the semantic segmentation of high-resolution hyperspectral remote sensing images. Our experiments on a simulated data set demonstrated the benefits of our hybrid model against conventional machine learning models in terms of extrapolation capabilities and interpretability. In particular, we show that p$^3$VAE naturally has high disentanglement capabilities. Our code and data have been made publicly available at https://github.com/Romain3Ch216/p3VAE.Comment: 21 pages, 11 figures, submitted to the International Journal of Computer Visio

Clay Minerals Mapping from Imaging Spectroscopy

Mapping subsurface clay minerals is an important issue because they have particular behaviors in terms of mechanics and hydrology that directly affects assets laid at the surface such as buildings, houses, etc. They have a direct impact in ground stability due to their swelling capacities, constraining infiltration processes during flooding, especially when moisture is important. So detecting and characterizing clay mineral in soils serve urban planning issues and improve the risk reduction by predicting impacts of subsidence on houses and infrastructures. High-resolution clay maps are thus needed with accurate indications on mineral species and abundances. Clay minerals, known as phyllosilicates, are divided in three main species: smectite, illite, and kaolinite. The smectite group highly contributes to the swelling behavior of soils, and because geotechnical soil analyses are expensive and time-consuming, it is urgent to develop new approaches for mapping clays’ spatial distribution by using new technologies, e.g., ground spectrometer or remote hyperspectral cameras [0.4–2.5 μm]. These technics constitute efficient alternatives to conventional methods. We present in this chapter some recent results we got for characterizing clay species and their abundances from spectrometry, used either from a ground spectrometer or from hyperspectral cameras

Toulouse Hyperspectral Data Set: a benchmark data set to assess semi-supervised spectral representation learning and pixel-wise classification techniques

Airborne hyperspectral images can be used to map the land cover in large urban areas, thanks to their very high spatial and spectral resolutions on a wide spectral domain. While the spectral dimension of hyperspectral images is highly informative of the chemical composition of the land surface, the use of state-of-the-art machine learning algorithms to map the land cover has been dramatically limited by the availability of training data. To cope with the scarcity of annotations, semi-supervised and self-supervised techniques have lately raised a lot of interest in the community. Yet, the publicly available hyperspectral data sets commonly used to benchmark machine learning models are not totally suited to evaluate their generalization performances due to one or several of the following properties: a limited geographical coverage (which does not reflect the spectral diversity in metropolitan areas), a small number of land cover classes and a lack of appropriate standard train / test splits for semi-supervised and self-supervised learning. Therefore, we release in this paper the Toulouse Hyperspectral Data Set that stands out from other data sets in the above-mentioned respects in order to meet key issues in spectral representation learning and classification over large-scale hyperspectral images with very few labeled pixels. Besides, we discuss and experiment the self-supervised task of Masked Autoencoders and establish a baseline for pixel-wise classification based on a conventional autoencoder combined with a Random Forest classifier achieving 82% overall accuracy and 74% F1 score. The Toulouse Hyperspectral Data Set and our code are publicly available at https://www.toulouse-hyperspectral-data-set.com and https://www.github.com/Romain3Ch216/tlse-experiments, respectively.Comment: 17 pages, 13 figure

Étude du potentiel des données hyperspectrales en vue de développer une méthode de cartographie automatique du patrimoine arboré en milieu urbain

Étude du potentiel des données hyperspectrales en vue de développer une méthode de cartographie automatique du patrimoine arboré en milieu urbain

Detection of individual trees in urban alignment from airborne data and T contextual information: A marked point process approach

With the current expansion of cities, urban trees have an important role for preserving the health of its in- habitants. With their evapotranspiration, they reduce the urban heat island phenomenon, by trapping CO2 emission, improve air quality. In particular, street trees or alignment trees, create shade on the road network, are structuring elements of the cities and decorate the roads. Street trees are also subject to specific conditions as they have little space for growth, are pruned and can be affected by the spread of diseases in single-species plantations. Thus, their detection, identification and monitoring are necessary. In this study, an approach is proposed for mapping these trees that are characteristic of the urban environment. Three areas of the city of Toulouse in the south of France are studied. Airborne hyperspectral data and a Digital Surface Model (DSM) for high vegetation detection are used. Then, contextual information is used to identify the street trees. Indeed, Geographic Information System (GIS) data are considered to detect the vegetation canopies close to the streets. Afterwards, individual street tree crown delineation is carried out by modeling the discriminative contextual features of individual street trees (hypotheses of small angle between the trees and similar heights) based on Marked Point Process (MPP). Compared to a baseline individual tree crown delineation method based on region growing, our method logically provides the best results with F-score values of 91%, 75% and 85% against 70%, 41% and 20% for the three studied areas respectively. Our approach mainly succeeds in identifying the street trees. In addition, the contribution of the angle, the height and the GIS data in the street tree mapping has been studied. The results encourage the use of the angle, the height and the GIS data together. However, with only the angle and the height, the results are similar to those obtained with the inclusion of the GIS data for the first and the second study cases with F-score values of 88%, 79% and 62% against 91%, 75% and 85% for the three study cases respectively. Finally, it is shown that the GIS data only is not sufficient

Feasibility Study for an Aquatic Ecosystem Earth Observing System Version 1.2.

International audienceMany Earth observing sensors have been designed, built and launched with primary objectives of either terrestrial or ocean remote sensing applications. Often the data from these sensors are also used for freshwater, estuarine and coastal water quality observations, bathymetry and benthic mapping. However, such land and ocean specific sensors are not designed for these complex aquatic environments and consequently are not likely to perform as well as a dedicated sensor would. As a CEOS action, CSIRO and DLR have taken the lead on a feasibility assessment to determine the benefits and technological difficulties of designing an Earth observing satellite mission focused on the biogeochemistry of inland, estuarine, deltaic and near coastal waters as well as mapping macrophytes, macro-algae, sea grasses and coral reefs. These environments need higher spatial resolution than current and planned ocean colour sensors offer and need higher spectral resolution than current and planned land Earth observing sensors offer (with the exception of several R&D type imaging spectrometry satellite missions). The results indicate that a dedicated sensor of (non-oceanic) aquatic ecosystems could be a multispectral sensor with ~26 bands in the 380-780 nm wavelength range for retrieving the aquatic ecosystem variables as well as another 15 spectral bands between 360-380 nm and 780-1400 nm for removing atmospheric and air-water interface effects. These requirements are very close to defining an imaging spectrometer with spectral bands between 360 and 1000 nm (suitable for Si based detectors), possibly augmented by a SWIR imaging spectrometer. In that case the spectral bands would ideally have 5 nm spacing and Full Width Half Maximum (FWHM), although it may be necessary to go to 8 nm wide spectral bands (between 380 to 780nm where the fine spectral features occur -mainly due to photosynthetic or accessory pigments) to obtain enough signal to noise. The spatial resolution of such a global mapping mission would be between ~17 and ~33 m enabling imaging of the vast majority of water bodies (lakes, reservoirs, lagoons, estuaries etc.) larger than 0.2 ha and ~25% of river reaches globally (at ~17 m resolution) whilst maintaining sufficient radiometric resolution